The Promise of Load-Balancing the Parameterization of Moist Convection using a Model Data Load Index

The parameterization of physical processes in Atmospheric General Circulation Models contribute to load imbalances among individual processors of message passing distributed multiprocessor systems. Load imbalances increase the overall time to completion of a model run and should be eliminated or reduced as much as possible. Presented is a new technique which shows promise for load-balancing the parameterization of moist convection found in Community Climate System Model (CCSM) Community Atmosphere Model (CAM3). The technique utilizes a load index that is a marker for moist convection (called a model data load index). The marker for moist convection correlates directly to the amount of processing time per model grid cell and can therefore be used to effect a load balance. Spatial locality on the model grid and temporal locality between model time-steps exist that allow a decomposition from a load balancing step to be retained for multiple time-steps. The analysis in this paper shows that the load balance does not need to be applied at every time-step and that the number of steps in which the previous load balance remains effective is large enough for the overhead to be cost effective. Tests performed indicate that this technique is scalable to higher resolution models as well as to higher processor counts than those presented. Through the use of the Load Balancing and Scheduling Framework (LBSF), this technique shows promise in reducing the time due to load imbalance of one particular subroutine of CAM3 by ∼47% at the T85 spectral truncation. A maximum of 3.75 seconds of total execution time is saved over a 2430 time-step simulation. When extrapolated to a 1000 year simulation, this translates to a possible savings of ∼22 hours in one subroutine alone. Similar methods applied to remaining subroutines can add up to a significant savings. These results are encouraging in that a fine-grained load-balancing technique using the evolving characteristics of geophysical data paves the way for load-balancing a broad range of physical calculations, both in CAM3 and other scientific applications, where more general techniques are not practical.